Image processor, imaging processing method, and program

ABSTRACT

There are provided an image processor, an image processing method, and a program that acquire visible pixel information regarding a pixel viewed by a user in an image and perform blur processing on a basis of the visible pixel information and depth information indicating a depth value corresponding to each of pixels of the image during a predetermined period after the visible pixel information is acquired, thereby further enhancing a sense of immersion as if in a different space.

TECHNICAL FIELD

The present disclosure relates to an image processor, an imageprocessing method, and a program.

BACKGROUND ART

In recent years, various techniques have been proposed in which ahead-tracking sensor is mounted on an HMD (Head Mounted Display) andimages generated (rendered) from a viewpoint depending on a position ofa head and a posture of a user are displayed on the HMD (e.g., PTL 1listed below).

The technique that enables the user to view a content while freelychanging the viewpoint in this manner is also called a free viewpointimage technique, which is able to give the user a sense of immersion asif in a different space.

CITATION LIST Patent Literature

PTL 1: Japanese Unexamined Patent Application Publication No.2016-025633

SUMMARY OF THE INVENTION Problem to be Solved by the Invention

In the technique described above, it has been desired to further enhancea sense of immersion as if in a different space.

Means for Solving the Problem

According to the present disclosure, there is provided an imageprocessor including: an acquisition section that acquires visible pixelinformation regarding a pixel viewed by a user in an image; and an imageprocessing section that performs blur processing on the image on a basisof depth information indicating a depth value corresponding to each ofpixels of the image, in which the image processing section performs theblur processing on a basis of the visible pixel information and thedepth information during a predetermined period after the visible pixelinformation is acquired.

In addition, according to the present disclosure, there is provided animage processing method including: acquiring visible pixel informationregarding a pixel viewed by a user in an image; and causing a processorto perform blur processing on the image on a basis of depth informationindicating a depth value corresponding to each of pixels of the image,in which the blur processing is performed on the pixel viewed by theuser in the image on a basis of the visible pixel information and thedepth information during a predetermined period after the visible pixelinformation is acquired.

In addition, according to the present disclosure, there is provided aprogram that causes a computer to function as an image processor, inwhich the image processor includes an acquisition section that acquiresvisible pixel information regarding a pixel viewed by a user in animage, and an image processing section that performs blur processing onthe image on a basis of depth information indicating a depth valuecorresponding to each of pixels of the image, in which the imageprocessing section performs the blur processing on a basis of thevisible pixel information and the depth information during apredetermined period after the visible pixel information is acquired.

Effect of the Invention

As described above, according to the present disclosure, it is possibleto further enhance a sense of immersion as if in a different space.

It is to be noted that the above-mentioned effects are not necessarilylimitative; in addition to or in place of the above effects, there maybe achieved any of the effects described in the present specification orother effects that may be grasped from the present specification.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is an explanatory diagram illustrating an outline of processingaccording to a first embodiment of the present disclosure.

FIG. 2 is a block diagram illustrating an example of a configuration ofa display system according to the same embodiment.

FIG. 3 is an explanatory diagram that describes a mechanism of blurprocessing occurring in imaging of a camera.

FIG. 4 is an explanatory diagram that describes intensity of blurprocessing performed by an image processing section 249.

FIG. 5 illustrates an example of blurring filters.

FIG. 6 is a flowchart diagram illustrating a flow of processing of adisplay system 1000 according to the present embodiment.

FIG. 7 is a flowchart diagram illustrating a detailed flow of processingof step S160.

FIG. 8 is a block diagram illustrating an example of a configuration ofa display system according to a second embodiment of the presentdisclosure.

FIG. 9 is a flowchart diagram illustrating a detailed flow of processingof step S160 in the same embodiment.

FIG. 10 is a block diagram illustrating an example of a configuration ofa display system 3000 according to Modification Example 1.

FIG. 11 is an explanatory diagram of an example in which a visible pixelis specified using information indicating a dominant eye.

FIG. 12 is an explanatory diagram illustrating a hardware configurationexample.

MODES FOR CARRYING OUT THE INVENTION

Hereinafter, description is given in detail of preferred embodiments ofthe present disclosure with reference to the accompanying drawings. Itis to be noted that, in the present specification and drawings, repeateddescription is omitted for components substantially having the samefunctional configuration by assigning the same reference numerals.

In addition, there is a case where, in the present specification anddrawings, a plurality of components having substantially the samefunctional configurations may be distinguished by assigning differentalphabets that follow the same reference numerals. However, in a casewhere it is unnecessary to particularly distinguish among the pluralityof components having substantially the same functional configurations,only the same reference numerals are assigned.

It is to be noted that description is given in the following order.

<<1. First embodiment>>

<1-1. Overview>

<1-2. Configuration>

<1-3. Operation>

<1-4. Effects>

<<2. Second Embodiment>>

<2-1. Configuration>

<2-2. Operation>

<2-3. Specific Examples>

<2-4. Effects>

<<3. Modification Examples>>

<3-1. Modification Example 1>

<3-2. Modification Example 2>

<3-3. Modification Example 3>

<<4. Hardware Configuration Example>> <<5. Closing>> 1. First Embodiment1-1. Overview

Before describing a display system according to a first embodiment ofthe present disclosure, description is given first of a backgroundleading to creation of the display system according to the presentembodiment.

For realistic expressions in games, movies, and the like, or forexpressions that highlight a specific object, a portion of region issubjected to blur processing for displaying. Such image expression isconsidered to be effective also in a free viewpoint image technique.

In particular, in a case of displaying an image on an HMD worn on a headof a user and covering a field of view of the user, such imageexpression is considered to be effective in causing the user to feel asif reproducing a focusing mechanism in a human eye. In other words, ablurred region appears to be a region that is not focused by the user,and a non-blurred region appears to be a region that is focused by theuser.

Therefore, for example, by acquiring information regarding a pixelviewed by the user in the image (hereinafter, referred to as a visiblepixel), i.e., visible pixel information and by performing blurprocessing on the basis of the visible pixel information, effects areconsidered to be obtained, in which the user is caused to feel as ifreproducing the focusing mechanism in the human eye to enhance a senseof immersion. According to the free viewpoint image technique, it ispossible to acquire depth information (e.g., depth map) corresponding toa generated image. Therefore, it is considered, for example, that, adepth value corresponding to the visible pixel is acquired by referringto the depth information using the visible pixel information, and theblur processing is performed to a region having a depth value with alarge difference from the depth value corresponding to the visible pixelto thereby allow for the above-described effects.

Incidentally, the human eye has a feature in which it takes a certainamount of time to focus. Accordingly, when a movement of a line of sightof a user is reflected immediately in the blur processing based on thevisible pixel information described above, the user may feel a sense ofdiscomfort, leading to a possibility that the sense of immersion maydeteriorate.

Therefore, the first embodiment of the present disclosure has beencreated, with the above-described circumstance as a point ofobservation. According to the present embodiment, a depth valuecorresponding to the visible pixel of the user is delayed, and the blurprocessing is performed on the basis of the delayed depth value(hereinafter, also called a delay depth value), thereby making itpossible to further enhance the sense of immersion given to the user.Hereinafter, description is given of an outline of processing of thepresent embodiment having such effects.

FIG. 1 is an explanatory diagram illustrating an outline of processingaccording to the present embodiment. An input image illustrated in FIG.1 is an image generated by a free viewpoint image technique dependingon, for example, a position of a head and a posture of a user. Inaddition, depth information illustrated in FIG. 1 is informationindicating a depth value corresponding to each of pixels of the inputimage, and may be, for example, a depth map. In addition, the visiblepixel information illustrated in FIG. 1 may be, for example, informationindicating a position of the visible pixel in the input image.

As illustrated in FIG. 1, first, processing of referring to the depthinformation based on the visible pixel information is performed (S1),and a depth value corresponding to the visible pixel is outputted.Subsequently, delay processing for delaying the depth valuecorresponding to the visible pixels is performed (S2), and a delay depthvalue is outputted. Then, the blur processing is performed on the inputimage on the basis of the delay depth value (S3), and an output image isoutputted.

The description has been given above of the outline of the processing ofthe present embodiment. The output image outputted by the processingillustrated in FIG. 1 is displayed, for example, on an HMD worn on thehead of the user, and the user feels as if reproducing a focusingmechanism in the human eye. Hereinafter, description is givensequentially of configuration examples and operation examples of adisplay system according to the present embodiment that achieves suchprocessing and effects.

1-2. Configuration

FIG. 2 is a block diagram illustrating an example of a configuration ofthe display system according to the present embodiment. As illustratedin FIG. 2, a display system 1000 according to the present embodimentincludes an HMD 1 and an image processor 2-1.

(HMD)

The HMD 1 is a display apparatus worn on the head of the user. The HMD 1includes a sensor unit 12, a display unit 14, and a communication unit16 as illustrated in FIG. 2.

The sensor unit 12 acquires, by sensing, information regarding the userand a surrounding environment. The sensor unit 12 includes, for example,an acceleration sensor, a gyro sensor, and the like for acquiringinformation indicating the position of the head and the posture of theuser. In addition, the sensor unit 12 may include a line-of-sight sensorthat is able to acquire visible pixel information regarding a visiblepixel viewed by the user in an image displayed on the display unit 14.

The display unit 14 displays an output image received by thecommunication unit 16 from the image processor 2-1. The display unit 14may be configured to be able to display separate images on the left eyeand the right eye of the user. Such a configuration causes the displayunit 14 to display, for example, a so-called stereoscopic image havingbinocular parallax, thereby enabling the user to carry out binocularstereopsis. It is to be noted that the display unit 14 may separatelyinclude a left eye display and a right eye display, or may display aleft eye image on left side of one display and a right eye image onright side of the one display.

The communication unit 16 is a communication module for transmitting andreceiving data to and from another apparatus by wire or wirelessly. Thecommunication unit 16 performs wireless communication with an externalapparatus directly or via a network access point, for example, in amethod such as wired LAN (Local Area Network), wireless LAN, Wi-Fi(Wireless Fidelity, registered trademark), infrared communication,Bluetooth (registered trademark), or short-range/non-contactcommunication.

For example, the communication unit 16 transmits information indicatingthe position of the head and the posture of the user acquired by thesensor unit 12 and the visible pixel information of the user to theimage processor 2-1, and receives an output image from the imageprocessor 2-1.

(Image Processor)

The image processor 2-1 includes a communication unit 22, a control unit24-1, and a storage unit 26, as illustrated in FIG. 2.

The communication unit 22 is a communication module for transmitting andreceiving data to and from another apparatus by wire or wirelessly. Thecommunication unit 22 performs wireless communication with an externalapparatus directly or via a network access point, for example, in amethod such as wired LAN (Local Area Network), wireless LAN, Wi-Fi(Wireless Fidelity, registered trademark), infrared communication,Bluetooth (registered trademark), or short-range/non-contactcommunication.

For example, the communication unit 22 functions as an acquisition unit,and receives (acquires), from the HMD 1, information indicating theposition of the head and the posture of the user as well as the visiblepixel information of the user. In addition, the communication unit 22transmits, to the HMD 1, an output image outputted from the control unit24-1 described later.

The control unit 24-1 functions as an arithmetic processing device and acontrol device, and controls overall operations inside the imageprocessor 2-1 in accordance with various programs. In addition, thecontrol unit 24-1 according to the present embodiment functions as acommunication control section 241, an input image generation section243, a depth value acquisition section 245, a delay control section 247,and an image processing section 249, as illustrated in FIG. 2.

The communication control section 241 controls communication made by thecommunication unit 22. For example, the communication control section241 controls the communication unit 22 to transmit an output imageoutputted from the image processing section 249 to the HMD 1, therebycausing the HMD 1 to display the output image.

The input image generation section 243 generates an input image on thebasis of information indicating the position of the head and the postureof the user received by the communication unit 22 from the HMD 1 and onthe basis of content data stored in the storage unit 26, and providesthe generated input image to the image processing section 249. The inputimage generated by the input image generation section 243 may be animage that have been generated (rendered) from a viewpoint depending onthe position of the head and the posture of the user, as describedabove. In addition, the input image generation section 243 may generatea left eye input image and a right eye input image for stereoscopicdisplay on the HMD 1. However, there is no particular difference betweenthe left eye input image and the right eye input image in processingdescribed below, and thus the processing described below as beingperformed on an input image can be applied, each independently, to theleft eye input image and the right eye input image.

In addition, on the basis of the content data stored in the storage unit26, the input image generation section 243 generates depth informationcorresponding to a generated input image (e.g., a depth map indicatingdepth of each of pixels of the input image), and provides the generateddepth information to the depth value acquisition section 245 and theimage processing section 249.

On the basis of the visible pixel information received by thecommunication unit 22 from the HMD 1 and the depth information providedfrom the input image generation section 243, the depth value acquisitionsection 245 acquires a depth value corresponding to the visible pixel ofthe user, and outputs the acquired depth value to the delay controlsection 247.

As described above, the depth information and the input image areassociated with each other. In addition, the visible pixel informationis information indicating a visible pixel in the display unit 14 of theHMD 1. The relationship of the association between the input image andthe display unit 14 may be specified in advance. Therefore, for example,the depth value acquisition section 245 is able to specify a position ofthe visible pixel in the input image from the visible pixel informationand refer to the depth information using the association between thedepth information and the input image to acquire a depth valuescorresponding to the visible pixel.

The delay control section 247 delays the depth value corresponding tothe visible pixel acquired by the depth value acquisition section 245,and outputs the delayed delay depth value to the image processingsection 249. The delay processing performed by the delay control section247 is expressed by, for example, the following Expression (1).

D _(n) =kD _(n-1)+(1−k)d _(n)  (1)

d_(n): a depth value corresponding to a visible pixel at frame t=nD_(n): a depth value to be used for blur processing at frame t=n (delaydepth value)D_(n-1): a depth value used for blur processing at frame t=n−1 (pastdelay depth value)k: delay parameter

The delay control section 247 outputs a delay depth value D_(n) to theimage processing section 249, and causes the storage unit 26 to storethe delay depth value D_(n). Then, when the delay processing isperformed at the next frame, the delay control section 247 reads thepreviously outputted delay depth value from the storage unit 26, anduses the read delay depth value as the past delay depth value D_(n-1) inthe expression (1) to perform the delay processing.

In addition, in the expression (1), the delay parameter k is a parameterfor adjusting a delay degree; the smaller delay parameter k is, thesmaller the delay is. For example, when the delay parameter k=0 holdstrue, a depth value corresponding to the visible pixel is outputtedimmediately without delay. In addition, for example, when the delayparameter k=0.92 holds true, in a case where the visible pixel of theuser does not move, the delay depth value D_(n) reaches about 95% of thedepth value corresponding to the visible pixel at 44 frames; forexample, in a case of being displayed at 90 fps, time required thereforis about 0.5 seconds.

Providing a delay depth value to the image processing section 249described later in this manner delays reflection of the visible pixel toblur processing performed by the image processing section 249. Then, theuser feels as if reproducing a focusing mechanism in the human eye, thusachieving the effect of further enhancing the sense of immersion.

It is to be noted that, in the present embodiment, the delay parameter kmay be preset, for example, and stored in the storage unit 26 describedlater.

On the basis of the delay depth value outputted from the delay controlsection 247 and the depth information provided from the input imagegeneration section 243, the image processing section 249 performs theblur processing on the input image generated by the input imagegeneration section 243 to generate an output image. The delay depthvalue to be used by the image processing section 249 in the blurprocessing described below is delayed and outputted by the delay controlsection 247 as described above. For this reason, during a predeterminedperiod after visible pixel information is acquired (received) regardinga visible pixel viewed by the user in the image, the image processingsection 249 performs the blur processing also on the visible pixel. As aresult, a feature in which it takes a certain amount of time for thehuman eye to focus is reproduced, thus making it possible to furtherenhance the sense of immersion given to the user.

The blur processing to be performed by the image processing section 249on an input image may be, for example, processing for reproducingblurring occurring in an image acquired by imaging of a camera.Therefore, description is given of the mechanism of blurring occurringin the imaging of the camera with reference to FIG. 3. FIG. 3 is anexplanatory diagram that describes the mechanism of blurring occurringin the imaging of the camera.

FIG. 3 illustrates a state in which, an image of an object O, which is asubject, is formed at a position of a point 1, and an image of an objectO_(n) is formed at a position of a point l_(n). A distance a between apoint A and a point B illustrated in FIG. 3 is an aperture diameter (adiameter of a surface on which light is incident) of a lens. Inaddition, a distance c between a point A′ and a point B′ illustrated inFIG. 3 is a diameter of a circle of confusion which is blurring of animage produced by an object behind or in front of the object O (objectO_(n) in the example illustrated in FIG. 3). In addition, a distanced_(o) illustrated in FIG. 3 is a distance between the lens and theobject O. In addition, a distance d_(n) illustrated in FIG. 3 is adistance between the lens and the object O_(n) closer to the lens thanthe object O. In addition, d_(i) illustrated in FIG. 3 is a distancebetween the lens and an image sensor. d_(c) illustrated in FIG. 3 is adistance between the lens and the image of the object O_(n).

Here, in FIG. 3, a triangle created by the point l_(n), the point A′,and the point B′ and a triangle created by the point l_(n), the point A,and the point B are similar to each other, and thus the followingexpressions (2) and (3) hold true.

a:c=d _(c) :d _(c) −d _(i)  (2)

Cd _(c) =a(d _(c) −d _(i))  (3)

Further, when a focal distance of the lens is defined as f, on the basisof lens formula, the following holds true.

${d_{c} = \frac{d_{n}f}{d_{n} - f}},{d_{i} = \frac{d_{0}f}{d_{0} - f}}$

Accordingly, the expression (3) can be modified as follows.

$c = {a\frac{\frac{d_{n}f}{d_{n} - f} - \frac{d_{0}f}{d_{0} - f}}{\frac{d_{n}f}{d_{n} - f}}}$$c = {{af}\frac{d_{0} - d_{n}}{d_{n}\left( {d_{0} - f} \right)}}$

Further, an F value, which is an index indicating brightness of a lens,is a value obtained by dividing the focal distance f by the aperturediameter, and thus the following expression (4) is obtained.

$\begin{matrix}{c = {\frac{f^{2}}{F}\frac{d_{0} - d_{n}}{d_{n}\left( {d_{0} - f} \right)}}} & (4)\end{matrix}$

It is appreciated, from the expression (4), that a magnitude of thedistance c, which is a diameter of the circle of confusion, i.e., amagnitude of the blurring, is proportional to the square of the focaldistance f and is inversely proportional to the F value, in a case wherethe distance d_(O) between the lens and the object O is sufficientlylarger than the focal distance f.

On the basis of the mechanism of blurring in the camera as describedabove, the image processing section 249 according to the presentembodiment performs blur processing with intensity corresponding to adifference between the delay depth value outputted from the delaycontrol section 247 and the depth value corresponding to each of pixels.For example, in a case where there is a large difference between thedelay depth value and the depth value corresponding to a certain pixel,the image processing section 249 may perform the blur processing withstrong intensity on the pixel as compared with a case where there is asmall difference between the delay depth value and the depth valuecorresponding to the pixel. Such a configuration can reproduce effectssimilar to the blurring mechanism of the camera as described above withreference to FIG. 3.

FIG. 4 is an explanatory diagram that describes the intensity of theblur processing performed by the image processing section 249. FIG. 4illustrates three objects OBJ1 to OBJ3. Assuming here that a point P onthe object OBJ2 is a point corresponding to the delay depth value usedfor the blur processing, the image processing section 249 preferablyperforms blur processing on an input image as if the point P was infocus. For example, the image processing section 249 preferablyperforms, on the input image, such blur processing that the object OBJ2is included in depth of field. In addition, a distance D23 in adirection of depth Z between the object OBJ2 and the object OBJ3 islarger than a distance D12 in the direction of the depth Z between theobject OBJ 1 and the object OBJ2. Thus, it is desirable to perform theblur processing with stronger intensity on the pixel of the object OBJ3than on the pixel of the object OBJ1.

For example, the image processing section 249 may apply a blurringfilter such as Gaussian filter to each of pixels to perform the blurprocessing. In addition, in such a case, the image processing section249 is able to control the intensity of the blur processing by a taplength (filter size) of the blurring filter.

For example, the image processing section 249 may set the tap length ofthe blurring filter applied to each of the pixels in accordance with thedifference between the delay depth value and the depth valuecorresponding to each of the pixels; the larger the difference betweenthe delay depth value and the depth value corresponding to each of thepixels is, the larger the tap length may be set.

FIG. 5 illustrates an example of blurring filters. A blurring filter F1illustrated in FIG. 5 is a Gaussian filter with a tap length being setto 3×3, and is applied to pixels of the object OBJ1 illustrated in FIG.4. In addition, a blurring filter F2 illustrated in FIG. 5 is a Gaussianfilter with a tap length being set to 1×1, and is applied to pixels ofthe object OBJ2 illustrated in FIG. 4. In addition, a blurring filter F3illustrated in FIG. 5 is a Gaussian with a filter tap length being setto 5×5, and is applied to pixels of the object OBJ3 illustrated in FIG.4.

Applying the blurring filters F1 to F3 with different tap lengths inthis manner causes an output image to be outputted to allow the pixelsof the object OBJ2 to appear clear. In addition, in the output image,the object OBJ1 slightly distant from the object OBJ2 appears moreblurred than the object OBJ2, and the object OBJ3 more distant from theobject OBJ2 appears still more blurred than the object OBJ1. As aresult, the user feels as if reproducing the focusing mechanism in thehuman eye, thus achieving the effect of further enhancing the sense ofimmersion.

It is to be noted that the blur processing performed by the imageprocessing section 249 is not limited to such examples. For example, theimage processing section 249 may perform the blur processing using ablurring filter other than the Gaussian filter. In addition, the imageprocessing section 249 may perform well-known image processing otherthan the blur processing.

Returning to FIG. 2, the description is continued for the configurationof the image processor 2-1. The storage unit 26 stores programs forcausing the respective configurations of the image processor 2-1 tofunction and parameters. In addition, the storage unit 26 may storecontent data for the input image generation section 243 to generate aninput image and depth information corresponding to the input image Thecontent data stored in the storage unit 26 may be, for example, acombination of images and the depth information, or a combination ofmesh polygon data and textures.

1-3. Operation

The description has been given above of the configuration example of thedisplay system 1000 according to the present embodiment. Consequently,description is given of an operation example of the display system 1000according to the present embodiment with reference to FIGS. 6 and 7.

FIG. 6 is a flowchart diagram illustrating a flow of processing of thedisplay system 1000 according to the present embodiment. As illustratedin FIG. 6, first, the sensor unit 12 of the HMD 1 acquires, by sensing,a position of the head and a posture of a user (S120). Subsequently, thesensor unit 12 of the HMD 1 acquires, by sensing, a visible pixel of theuser (S140). It is to be noted that information on the position of thehead and the posture of the user acquired in step S120 and visible pixelinformation regarding the visible pixel of the user acquired in the stepS140 are transmitted from the HMD 1 to the image processor 2-1 asoccasion demands.

The image processor 2-1 generates an output image on the basis ofinformation received from the HMD 1 (S160). Details of processing ofstep S160 are described later with reference to FIG. 7. Subsequently,the output image generated in step S160 is transmitted from the imageprocessor 2-1 to the HMD 1, and is displayed on the display unit 14 ofthe HMD 1 (S180).

FIG. 7 is a flowchart diagram illustrating a detailed flow of theprocessing of step S160 illustrated in FIG. 6. As illustrated in FIG. 7,first, the input image generation section 243 generates an input imageand depth information corresponding to the input image on the basis ofinformation indicating the position of the head and the posture of theuser acquired in step S120 and on the basis of the content data storedin the storage unit 26 (S161).

Subsequently, the depth value acquisition section 245 refers to thedepth information corresponding to the input image generated in stepS161 to acquire a depth value corresponding to the visible pixelacquired in step S140 (S161).

Subsequently, the delay control section 247 reads a depth value used inprevious blur processing from the storage unit 26 (S163). Further, onthe basis of the depth value used in the previous blur processing readin step S163, the delay control section 247 delays the depth valuecorresponding to the visible pixel to output a delay depth value (S164).

The delay depth value outputted in step S164 is stored in the storageunit 26 as a depth value to be used in the current blur processing(S165). Then, the image processing section 249 performs blur processingon the input image on the basis of the delay depth value outputted instep S164 to generate an output image (S166).

1-4. Effects

As described above, according to the first embodiment of the presentdisclosure, reflection of the movement of the line of sight of the useris delayed, and the blur processing is performed on the basis of thevisible pixel information, thereby further enhancing the sense ofimmersion while suppressing the sense of discomfort given to the user.

2. Second Embodiment

The description has been given above of the first embodiment of thepresent disclosure. Subsequently, description is given of a secondembodiment of the present disclosure. In the foregoing first embodiment,the delay parameter k for adjusting the delay degree is preset. Incontrast, according to the second embodiment of the present disclosure,a trigger is detected on the basis of a predetermined condition, and, onthe basis of a detection result of the trigger, the delay parameter k ischanged to thereby control the delay degree of the depth value. Such aconfiguration makes it possible to provide the user with morecomfortable viewing or to further reduce the sense of discomfort givento the user.

Hereinafter, description is given of a configuration example of thesecond embodiment that achieves the above-described effects. It is to benoted that, in the following description, differences from the firstembodiment are mainly described, and descriptions of configurations andoperations that are common to those of the first embodiment are omittedas appropriate.

2-1. Configuration

FIG. 8 is a block diagram illustrating an example of a configuration ofa display system according to the present embodiment. As illustrated inFIG. 8, a display system 2000 according to the present embodimentincludes the HMD 1 and an image processor 2-2. It is to be noted that,among the configurations illustrated in FIG. 8, substantially the sameconfigurations as the configurations described with reference to FIG. 2are denoted by the same reference numerals.

As illustrated in FIG. 8, the image processor 2-2 includes thecommunication unit 22, a control unit 24-2, and the storage unit 26. Theconfigurations of the communication unit 22 and the storage unit 26illustrated in FIG. 8 are substantially the same as the configurationsof the communication unit 22 and the storage unit 26 described withreference to FIG. 2, and thus descriptions thereof are omitted here.

Similarly to the control unit 24-1 described with reference to FIG. 2,the control unit 24-2 functions as an arithmetic processing device and acontrol device, and controls overall operations inside the imageprocessor 2-2 in accordance with various programs. However, the controlunit 24-2 according to the present embodiment differs from the controlunit 24-1 described with reference to FIG. 2 in that the control unit24-2 also functions as a detection section 246 as illustrated in FIG. 8and in that a function of a delay control section 248 differs partiallyfrom the function of the delay control section 247.

The detection section 246 detects a trigger on the basis of apredetermined condition. The predetermined condition is a conditionrelated to a control of a delay degree of a depth value performed by thedelay control section 248 described later. For example, in a case whereno trigger is detected, the delay parameter k may be set to apredetermined reference value. Then, the detection section 246 detects atrigger for which the delay parameter k is desirably set to a valuesmaller than the reference value or a value larger than the referencevalue.

For example, the detection section 246 may detect a trigger related to astate of the user. Although the state of the user to be detected mayvary widely, for example, the detection section 246 may detect a triggerrelated to a speed of a movement of the user, or may detect a triggerrelated to a predetermined motion. It is to be noted here that themovement of the user may include not only the movement of the head ofthe user, but also the movement of a line of sight of the user.

In addition, the detection section 246 may detect a trigger related toan object included in an input image. For example, the detection section246 may detect a trigger related to a distance from the user to theobject. Alternatively, the detection may be performed for a triggerrelated to whether or not object is to be emphasized. It is to be notedthat the distance from user to the object is specified from depthinformation corresponding to the input image. In addition, the objectused to detect a trigger may be every object included in the inputimage, or may be an object viewed by the user or being present in thevicinity of the visible pixel of the user, among objects included in theinput image.

In addition, the detection section 246 may detect a trigger related to acontent type of the input image. The content type may include, forexample, a genre of a content. Alternatively, the content type mayinclude a type of whether or not an object of attention is clear.Alternatively, the content type may include a type of whether or not thecontent includes an object with a movement.

It is to be noted that, in order to detect the triggers as describedabove, the detection section 246 may use the information regarding theposition of the head and the posture of the user received from the HMD1, the visible pixel information, information regarding the contentstored in the storage unit 26, and other types of information. Inaddition, the detection of a trigger performed by the detection section246 may vary widely, and is not limited to the examples described above.

Similarly to the delay control section 247 described with reference toFIG. 2, the delay control section 248 delays the depth valuecorresponding to the visible pixel acquired by the depth valueacquisition section 245, and outputs the delayed delay depth value tothe image processing section 249. However, the delay control section 248according to the present embodiment differs from the delay controlsection 247 in that the delay degree of the depth value is controlled onthe basis of the detection result of the trigger performed by thedetection section 246 described above.

In a case where no trigger is detected by the detection section 246, thedelay control section 248 according to the present embodiment sets thedelay parameter k to a predetermined reference value, and performs thedelay processing in accordance with the above-described expression (1).The reference value is desirably a value at which focusing is performedat such a speed at which a person normally views an object, and thereference value may be 0.92 in a case of being displayed at 90 fps, forexample, although the value depends on the frame rate, or the like ofthe HMD 1.

In addition, in a case where a trigger is detected by the detectionsection 246, the delay control section 248 sets the delay parameter k toa value smaller than the reference value (a value close to 0) or a valuelarger than the reference value (a value close to 1), depending on thetype of the detected trigger. It is to be noted that, setting the valueof the delay parameter to a value smaller than the reference valuedecreases the delay, and setting the value of the delay parameter to avalue larger than the reference value increases the delay.

For example, in a case where the detection section 246 detects, as atrigger, that the movement of the user is faster than a predeterminedthreshold value, or in a case where the detection section 246 detects,as a trigger, a predetermined motion, such as the user walking around orthe user shaking his or her head, it is highly possible that theposition of the visible pixel changes at a high speed. Therefore, in acase where such a trigger is detected, the delay control section 248 mayset the value of the delay parameter k to a value smaller than thereference value to be able to follow the movement of the visible pixel.

In addition, in a case where the detection section 246 detects, as atrigger, that an object included in the input image is distant from theuser, it is highly possible that a change in the depth valuecorresponding to the visible pixel is small. Therefore, in a case wheresuch a trigger is detected, the delay control section 248 may set thevalue of the delay parameter k to a value smaller than the referencevalue to allow the user to feel smooth focusing.

In addition, the detection section 246 detects, as a trigger, that theuser views an object to be highlighted, it is desirable to highlight theobject to allow the user to pay attention to the object, even when thevisible pixel leaves the object thereafter. Therefore, in a case wheresuch a trigger is detected, the delay control section 248 may set thevalue of the delay parameter k to a value larger than the referencevalue to allow the user to pay attention to the object to behighlighted.

2-2. Operation

The description has been given above of the configuration example of thedisplay system 2000 according to the present embodiment. Subsequently,description is given of an operation example of the display system 2000according to the present embodiment. However, the flow of processing ofthe display system 2000 according to the present embodiment is similarto the flow of the processing of the display system 1000 described withreference to FIG. 6, except that the processing of step S160 illustratedin FIG. 6 is partially different. Therefore, a detailed flow of theprocessing of step S160 in the present embodiment is described withreference to FIG. 9.

FIG. 9 is a flowchart diagram illustrating a detailed flow of theprocessing of step S160 in the present embodiment. Processing of each ofsteps S171 to S173 illustrated in FIG. 9 is similar to the processing ofeach of steps S161 to S163 described with reference to FIG. 7, and thusdescriptions thereof are omitted here.

In the following step S174, the detection section 246 detects a trigger,and the delay control section 248 sets the delay parameter k on thebasis of the detection result of the trigger. Further, in accordancewith the delay parameter k set in step S174, and on the basis of thedepth value used in the previous blur processing read in step S173, thedelay control section 248 delays the depth value corresponding to thevisible pixel, and outputs the delay depth value (S175).

Processing of each of subsequent steps S176 to S177 illustrated in FIG.9 is similar to the processing of each of steps S165 to S166 describedwith reference to FIG. 7, and thus description thereof is omitted here.

2-3. Specific Examples

The description has been given above of the configuration example andthe operation example of the present embodiment. Hereinafter,description is given of some specific application examples of thepresent embodiment.

Specific Example 1

Description is given, as a specific example 1, of a case of viewing acontent in which a target of attention is clear.

For example, in a case of viewing a content in which a singer is aloneon a stage in a music live show, display is performed basically on thebasis of a delayed delay depth value, with a reference value being setas the delay parameter k. However, in a case where the detection section246 detects a high point of the music as a trigger, the delay parameterk larger than the reference value is set to allow the delay to increasewhen the visible pixel moves to an object other than the singer. Such aconfiguration makes it possible to present to the user the singer to bepaid attention to.

In addition, also in a case of watching a sports content such as soccer,for example, display is performed basically on the basis of a delayeddelay depth value, with a reference value being set as the delayparameter k. However, in a case where the detection section 246 detectsan exciting scene such as a goal scene as a trigger, the delay parameterk larger than the reference value is set to allow the delay to increasewhen movement is made to an unimportant position. Such a configurationmakes it possible to present the user an important scene to be paidattention to. Alternatively, in a case where the detection section 246detects, as a trigger, a motion of the user shaking his or her head, thedelay parameter k smaller than the reference value is set. Such aconfiguration allows the user to have a clear view, thus making iteasier for the user to view the entire image.

Specific Example 2

Description is given, as a specific example 2, of a case in which theuser is able to freely move around to view a content while changingviewpoints.

For example, in a case where the user is taking an extensive viewwithout moving much, display is performed on the basis of a delayeddelay depth value, with a reference value being set as the delayparameter k in order to achieve a more realistic expression. Forexample, this is more effective in a case of stereoscopically viewing amoving image in which nature is shot.

Meanwhile, in a case where the detection section 246 detects, as atrigger, that the user is moving around at a high speed in an FPS (FirstPerson Shooting) game or the like, the delay parameter k smaller thanthe reference value is set in order to quickly judge the situation fromvisual information.

In addition, in a case where the user views the image while moving, apredetermined period during which the blur processing is performed onthe visible pixel may be decreased as compared with a case where theuser views the image while stopping. Such a configuration enables theuser to view the image clearly even in a case where the user is moving.However, such an example is not limitative; in the case where the userviews the image while moving, the predetermined period during which theblur processing is performed on the visible pixel may be increased ascompared with the case where the user views the image while stopping.

2-4. Effects

As described above, according to the second embodiment of the presentdisclosure, a trigger is detected on the basis of a predeterminedcondition to control the delay degree of a depth value on the basis of adetection result of the trigger, thereby making it possible to providethe user with more comfortable viewing and to further reduce the senseof discomfort given to the user.

3. Modification Examples

The description has been given above of the embodiments of the presentdisclosure. Hereinafter, description is given of some modificationexamples according to the present disclosure. It is to be noted thatmodification examples described below may be applied to the embodimentsalone or in combination. In addition, the modification examples may beapplied in place of or in addition to the configurations described inthe foregoing embodiments.

3-1. Modification Example 1

In the foregoing embodiments, the descriptions have been given above ofthe examples, in each of which the display system is configured by twoapparatuses, i.e., the HMD and the image processor. However, the presenttechnology is not limited to such examples. For example, the HMD and theimage processor may be an integrated apparatus. Alternatively, there maybe another apparatus that provides content data. In particular, in acase where the content data includes mesh polygon data and texture data,the display system desirably includes another apparatus that providescontent data in terms of data volume. Hereinafter, description is given,as Modification Example 1, of an example in which the display systemincludes another apparatus that provides content data, in addition tothe HMD and the image processor.

FIG. 10 is a block diagram illustrating an example of a configuration ofa display system 3000 according to Modification Example 1. Asillustrated in FIG. 10, the display system 3000 according to the presentembodiment includes the HMD 1, an image processor 2-3, a server 4, and acommunication network 5. The image processor 2-3 and the server 4 arecoupled to each other by the communication network 5, and are able tocommunicate information to each other.

It is to be noted that, among the configurations illustrated in FIG. 10,substantially the same configurations as the configurations describedwith reference to FIG. 2 are denoted by the same reference numerals.

As illustrated in FIG. 19, the image processor 2-3 includes acommunication unit 23, a control unit 24-3, and a storage unit 27.

In addition to the functions of the communication unit 22 described withreference to FIG. 2, the communication unit 23 has a function ofcommunicating with the server 4 via the communication network 5. Forexample, the communication unit 23 transmits the information indicatingthe position of the head and the posture of the user received from theHMD 1 to the server 4, and receives mesh polygon data and texture datafrom the server 4.

The control unit 24-3 differs from the control unit 24-1 illustrated inFIG. 2 in that the functions of a communication control section 242 andan input image generation section 244 differ partially from thefunctions of the communication control section 241 and the input imagegeneration section 243 described with reference to FIG. 2.

The communication control section 242 differs from the communicationcontrol section 241 in that the communication control section 242controls the communication unit 23 to transmit to the server 4 theinformation indicating the position of the head and the posture of theuser and to receive from the server 4 the mesh polygon data and thetexture data.

The input image generation section 244 differs from the input imagegeneration section 243 in that the input image generation section 244uses the mesh polygon data and the texture data received by thecommunication unit 23 from the server 4 to generate an input image anddepth information corresponding to the input image.

The storage unit 27 differs from the storage unit 26 described withreference to FIG. 2 in that the storage unit 27 does not need to storecontent data in advance.

As illustrated in FIG. 10, the server 4 is an information processorincluding a communication unit 42, a control unit 44, and a storage unit46.

The communication unit 42 is a communication module for transmitting andreceiving data to and from another apparatus by wire or wirelessly. Thecommunication unit 16 performs wireless communication with an externalapparatus directly or via a network access point, for example, in amethod such as wired LAN (Local Area Network), wireless LAN, Wi-Fi(Wireless Fidelity, registered trademark), infrared communication,Bluetooth (registered trademark), or short-range/non-contactcommunication.

For example, the communication unit 42 communicates with the imageprocessor 2-3 via the communication network 5 to receive the informationindicating the position of the head and the posture of the user from theimage processor 2-3 and to transmit the mesh polygon data and thetexture data to the image processor 2-3.

The control unit 44 functions as an arithmetic processing device and acontrol device, and controls overall operations inside the server 4 inaccordance with various programs. In addition, the control unit 44selects a texture corresponding to a viewpoint depending on the positionof the head and the posture of the user from among a plurality oftextures stored in storage unit 46, and causes the communication unit 42to transmit the selected texture to the image processor 2-3.

The storage unit 46 stores content data including the mesh polygon dataand data of the plurality of textures.

The description has been given above of Modification Example 1.According to Modification Example 1, for example, even in a case where astorage capacity of the storage unit 27 included in the image processor2-3 is small or a case where processing performance is low, appropriatetexture data is received from the server 4, thus enabling smoothdisplay.

3-2. Modification Example 2

The description has been given, in the foregoing embodiments, of theexample in which the sensor unit 12 of the HMD 1 includes theline-of-sight sensor and the visible pixel information is acquired bysensing; however, the present technology is not limited to such anexample. For example, even in a case where the HMD 1 does not includethe line-of-sight sensor, it is possible to specify a visible pixel.Such an example is described as Modification Example 2.

For example, the visible pixel information may be specified on the basisof information indicating a dominant eye of the user. For example, it ispossible to specify, as the visible pixel, a position in the input imagecorresponding to the front position of the dominant eye of the user. Itis to be noted that, as used herein, the dominant eye is an eye on sidethat is better used by a user, or is an eye on side that is morepreferably used by the user. The information indicating the dominant eyemay be provided in advance or may be inputted by the user. Thespecifying of the visible pixel based on the information indicating thedominant eye may be performed by the HMD or by the image processor.

FIG. 11 is an explanatory diagram of an example in which a visible pixelis specified using the information indicating the dominant eye. FIG. 11illustrates a left eye display 14L and a right eye display 14R includedin the display unit 14 of the HMD 1. In a case where the dominant eye ofthe user is a left eye E_(L), the visible pixel is specified as a pointG_(L) in the left eye display 14L. In a case where the dominant eye ofthe user is a right eye E_(R), the visible pixel is specified as a pointG_(R) in the right eye display 14R. It is to be noted that, associationbetween a position in each display and an input image is achievable bywell-known techniques, and thus description thereof is omitted here.

The description has been given above of Modification Example 2.According to Modification Example 2, even in a case where the HMD 1 doesnot include the line-of-sight sensor, it is possible to specify thevisible pixel.

3-3. Modification Example 3

In the foregoing embodiments, the HMD is used as display apparatus, thepresent technology is not limited to such an example; it is possible toapply the present technology to the image display on a variety ofdisplay apparatuses. For example, applying the present technology to adisplay apparatus including a display unit that covers most of a fieldof view of the user, such as a dome-like display, also makes it possiblefor the user to obtain a higher sense of immersion, and is considered tobe effective.

4. Hardware Configuration Example

The description has been given above of the embodiments of the presentdisclosure. Finally, description is given of a hardware configuration ofan information processor according to the embodiments of the presentdisclosure with reference to FIG. 12. FIG. 12 is a block diagramillustrating an example of a hardware configuration of the informationprocessor according to embodiments of the present disclosure. It is tobe noted that an information processor 900 illustrated in FIG. 12 mayimplement, for example, the HMD 1, the image processors 2-1 to 2-3, andthe server 4 illustrated in FIGS. 2, 8, and 10, respectively.Information processing performed by the HMD 1, the image processors 2-1to 2-3, and the server 4 according to the embodiments of the presentdisclosure is achieved by cooperation of software together with hardwaredescribed below.

As illustrated in FIG. 12, the information processor 900 includes a CPU(Central Processing Unit) 901, a ROM (Read Only Memory) 902, a RAM(Random Access Memory) 903, and a host bus 904 a. In addition, theinformation processor 900 includes a bridge 904, an external bus 904 b,an interface 905, an input device 906, an output device 907, a storagedevice 908, a drive 909, a coupling port 911, a communication device913, and a sensor 915. The information processor 900 may include aprocessing circuit such as a DSP or an ASIC in place of or in additionto the CPU 901.

The CPU 901 functions as an arithmetic processing device and a controldevice, and controls overall operations inside the information processor900 in accordance with various programs. In addition, the CPU 901 may bea microprocessor. The ROM 902 stores a program to be used by the CPU901, arithmetic parameters, and the like. The RAM 903 temporarily storesa program to be used in execution by the CPU 901 and parameters thatchange appropriately in the execution. The CPU 901 may form, forexample, the control units 24-1 to 24-3 and the control unit 44.

The CPU 901, the ROM 902 and the RAM 903 are coupled to one another bythe host bus 904 a including a CPU bus. The host bus 904 a is coupled tothe external bus 904 b such as a PCI (Peripheral ComponentInterconnect/Interface) bus via the bridge 904. It is to be noted thatthe host bus 904 a, the bridge 904 and the external bus 904 b may notnecessarily be configured to be separated, but these functions may beimplemented in one bus.

The input device 906 is implemented by a device with which informationis inputted by a user, such as a mouse, a keyboard, a touch panel, abutton, a microphone, a switch, and a lever. In addition, the inputdevice 906 may be, for example, a remote control device using infraredrays or another radio wave, or may be an external coupling apparatussuch as a mobile phone or a PDA corresponding to operations of theinformation processor 900. Further, the input device 906 may include,for example, an input control circuit that generates an input signal onthe basis of information inputted by a user using the above-mentionedinput means and outputs the generated input signals to the CPU 901. Theuser of the information processor 900 is able to input various types ofdata or instruct processing operations to the information processor 900by operating this input device 906.

The output device 907 is formed by a device that is able to visually oraudibly notify the user of acquired information. Examples of such adevice include display devices such as a CRT display device, a liquidcrystal display device, a plasma display device, an EL display deviceand lamp, sound output devices such as speaker and a headphone, and aprinter device. The output device 907 outputs, for example, resultsobtained by various types of processing performed by the informationprocessor 900. Specifically, the display device visually displays theresults obtained by the various types of processing performed by theinformation processor 900 in a variety of formats, such as a text, animage, a table, and a graph. Meanwhile, the sound output device convertsan audio signal, including reproduced sound data or acoustic data, intoan analog signal and outputs the converted analog signal audibly. Theoutput device 907 may form the display unit 14, for example.

The storage device 908 is a device for storing data formed as an exampleof the storage unit of the information processor 900. The storage device908 is implemented by, for example, a magnetic storage unit device suchas an HDD, a semiconductor storage device, an optical storage device, amagneto-optical storage device, or the like. The storage device 908 mayinclude a storage medium, a recording device that records data in thestorage medium, a reading device that reads the data from the storagemedium, a deleting device that deletes the data recorded in the storagemedium, and the like. The storage device 908 stores programs to beexecuted by the CPU 901, various data, various data acquired from theoutside, and the like. The above storage device 908 may form, forexample, the storage unit 26, the storage unit 27, and the storage unit46.

The drive 909 is a reader/writer for a storage medium, and is built inor externally attached to the information processor 900. The drive 909reads information recorded in an attached removable storage medium suchas a magnetic disk, an optical disk, a magneto-optical disk, or asemiconductor memory, and outputs the read information to the RAM 903.In addition, the drive 909 is also able to write information into theremovable storage medium.

The coupling port 911 is an interface to be coupled to an externalapparatus, and is a coupling port with an external apparatus that isable to transmit data by a USB (Universal Serial Bus), for example.

The communication device 913 is, for example, a communication interfaceformed by a communication device, etc. for coupling to a network 920.The communication device 913 is, for example, a communication card, etc.for wired or wireless LAN (Local Area Network), LTE (Long TermEvolution), Bluetooth (registered trademark), or WUSB (Wireless USB). Inaddition, the communication device 913 may be a router for opticalcommunication, a router for ADSL (Asymmetric Digital Subscriber Line), amodem for various types of communication, or the like. The communicationdevice 913 is able to transmit and receive signals or the like to andfrom the Internet or other communication apparatuses in accordance witha predetermined protocol such as TCP/IP, for example. The communicationdevice 913 may form, for example, the communication unit 22, thecommunication unit 23, and the communication unit 42.

The sensor 915 may be, for example, various sensors such as anacceleration sensor, a gyro sensor, a geomagnetic sensor, an opticalsensor, a sound sensor, a ranging sensor, and a force sensor. The sensor915 acquires information regarding a state of the information processor900 itself, such as a posture and a moving speed of the informationprocessor 900, and information regarding a surrounding environment ofthe information processor 900, such as brightness and noise around theinformation processor 900. In addition, the sensor 915 may include a GPSsensor that receives a GPS signal and measures the latitude, longitude,and altitude of the apparatus. The sensor 915 may form, for example, thesensor unit 12.

It is to be noted that the network 920 is a wired or wirelesstransmission path for information transmitted from an apparatus coupledto the network 920. For example, the network 920 may include a publicnetwork such as the Internet, a telephone network, a satellitecommunication network, and various types of LAN (Local Area Network)including Ethernet (registered trademark), WAN (Wide Area Network), andthe like. In addition, the network 920 may include a private networksuch as IP-VPN (Internet Protocol-Virtual Private Network).

The description has been given above of an example of the hardwareconfiguration that makes it possible to implement the functions of theinformation processor 900 according to an embodiment of the presentdisclosure. Each of the above-described components may be implementedusing general-purpose members, or may be implemented by hardwarespecialized in the functions of the respective components. Accordingly,it is possible to appropriately change hardware configurations to beutilized in accordance with a technical level at the time ofimplementing the embodiments of the present disclosure.

It is to be noted that it is possible to create a computer program forimplementing each function of the information processor 900 according toan embodiment of the present disclosure as described above and to mountthe computer program on a PC, etc. In addition, it is also possible toprovide a computer-readable recording medium in which such a computerprogram is stored. The recording medium is, for example, a magneticdisk, an optical disk, a magneto-optical disk, a flash memory, or thelike. In addition, the computer program described above may bedistributed via a network, for example, without using a recordingmedium.

5. Closing

As described above, according to the embodiments of the presentdisclosure, it is possible to further enhance a sense of immersion as ifin a different space.

Although the description has been given above in detail of preferredembodiments of the present disclosure with reference to the accompanyingdrawings, the technical scope of the present disclosure is not limitedto such examples. It is obvious that a person having ordinary skill inthe art of the present disclosure may find various alterations ormodifications within the scope of the technical idea described in theclaims, and it should be understood that these alterations andmodifications naturally come under the technical scope of the presentdisclosure.

For example, the steps in the foregoing embodiments need not necessarilybe processed in time series in the order described as the flowchartdiagram. For example, the steps in the processing of the foregoingembodiments may be processed in an order different from the orderdescribed as the flowchart diagram or may be processed in parallel.

In addition, the effects described herein are merely illustrative orexemplary, and are not limitative. That is, the technology according tothe present disclosure may achieve, in addition to or in place of theabove effects, other effects that are obvious to those skilled in theart from the description of the present specification.

It is to be noted that the technical scope of the present disclosurealso includes the following configurations.

(1)

An image processor including:

an acquisition section that acquires visible pixel information regardinga pixel viewed by a user in an image; and

an image processing section that performs blur processing on the imageon a basis of depth information indicating a depth value correspondingto each of pixels of the image,

the image processing section performing the blur processing on a basisof the visible pixel information and the depth information during apredetermined period after the visible pixel information is acquired.

(2)

The image processor according to (1), in which the image processingsection performs the blur processing on a basis of a delay depth valueand the depth information, the delay depth value being a delayed depthvalue corresponding to the pixel viewed by the user.

(3)

The image processor according to (2), in which the image processingsection performs the blur processing on each of the pixels in accordancewith a difference between the delay depth value and the depth valuecorresponding to each of the pixels.

(4)

The image processor according to (3), in which, in a case where there isa large difference between the delay depth value and a depth valuecorresponding to a pixel, the image processing section performs the blurprocessing on the pixel with stronger intensity, as compared with a casewhere there is a small difference between the delay depth value and thedepth value corresponding to the pixel.

(5)

The image processor according to (3) or (4), in which the imageprocessing section performs the blur processing by applying a blurringfilter to each of the pixels, and sets a tap length of the blurringfilter in accordance with the difference between the delay depth valueand the depth value corresponding to each of the pixels.

(6)

The image processor according to any one of (2) to (5), in which

the image processor further includes

-   -   a detection section that detects a trigger on a basis of a        predetermined condition, and    -   a delay control section that delays the depth value        corresponding to the pixel viewed by the user and outputs the        delay depth value, and

the delay control section controls a delay degree of the depth value ona basis of a result of the detection of the trigger performed by thedetection section.

(7)

The image processor according to (6), in which the detection sectiondetects a trigger related to a state of the user.

(8)

The image processor according to (7), in which the detection sectiondetects a trigger related to a speed of a movement of the user.

(9)

The image processor according to any one of (6) to (8), in which thedetection section detects a trigger related to an object included in theimage.

(10)

The image processor according to (9), in which the detection sectiondetects a trigger related to a distance between the user and the object.

(11)

The image processor according to any one of (6) to (10), in which thedetection section detects a trigger related to a content type of theimage.

(12)

The image processor according to any one of (1) to (11), furtherincluding a depth value acquisition section that acquires a depth valuecorresponding to the pixel viewed by the user in the image on a basis ofthe visible pixel information and the depth information.

(13)

The image processor according to any one of (1) to (12), in which thepixel viewed by the user in the image is specified on a basis ofinformation indicating a dominant eye of the user.

(14)

The image processor according to any one of (1) to (13), in which theimage is rendered from a viewpoint depending on a position or a postureof the user.

(15)

The image processor according to any one of (1) to (14), in which, in acase where the user views an image while moving, the predeterminedperiod is decreased as compared with a case where the user views animage while stopping.

(16)

An image processing method including:

acquiring visible pixel information regarding a pixel viewed by a userin an image; and

causing a processor to perform blur processing on the image on a basisof depth information indicating a depth value corresponding to each ofpixels of the image,

the blur processing being performed on the pixel viewed by the user inthe image on a basis of the visible pixel information and the depthinformation during a predetermined period after the visible pixelinformation is acquired.

(17)

A program that causes a computer to function as an image processor,

the image processor including

an acquisition section that acquires visible pixel information regardinga pixel viewed by a user in an image, and

an image processing section that performs blur processing on the imageon a basis of depth information indicating a depth value correspondingto each of pixels of the image,

the image processing section performing the blur processing on a basisof the visible pixel information and the depth information during apredetermined period after the visible pixel information is acquired.

REFERENCE NUMERALS LIST

-   1 HMD-   2-1 image processor-   4 server-   5 communication network-   24-1 control unit-   26 storage unit-   243 input image generation section-   245 depth value acquisition section-   246 detection section-   247 delay control section-   249 image processing section

1. An image processor comprising: an acquisition section that acquiresvisible pixel information regarding a pixel viewed by a user in animage; and an image processing section that performs blur processing onthe image on a basis of depth information indicating a depth valuecorresponding to each of pixels of the image, the image processingsection performing the blur processing on the pixel viewed by the userin the image on a basis of a delay depth value and the depth informationduring a predetermined period after the visible pixel information isacquired, the delay depth value being a delayed depth valuecorresponding to the pixel viewed by the user.
 2. (canceled)
 3. Theimage processor according to claim 1, wherein the image processingsection performs the blur processing on each of the pixels in accordancewith a difference between the delay depth value and the depth valuecorresponding to each of the pixels.
 4. The image processor according toclaim 3, wherein, in a case where there is a large difference betweenthe delay depth value and a depth value corresponding to a pixel, theimage processing section performs the blur processing on the pixel withstronger intensity, as compared with a case where there is a smalldifference between the delay depth value and the depth valuecorresponding to the pixel.
 5. The image processor according to claim 3,wherein the image processing section performs the blur processing byapplying a blurring filter to each of the pixels, and sets a tap lengthof the blurring filter in accordance with the difference between thedelay depth value and the depth value corresponding to each of thepixels.
 6. The image processor according to claim 1, wherein the imageprocessor further comprises a detection section that detects a triggeron a basis of a predetermined condition, and a delay control sectionthat delays the depth value corresponding to the pixel viewed by theuser and outputs the delay depth value, and the delay control sectioncontrols a delay degree of the depth value on a basis of a result of thedetection of the trigger performed by the detection section.
 7. Theimage processor according to claim 6, wherein the detection sectiondetects a trigger related to a state of the user.
 8. The image processoraccording to claim 7, wherein the detection section detects a triggerrelated to a speed of a movement of the user.
 9. The image processoraccording to claim 6, wherein the detection section detects a triggerrelated to an object included in the image.
 10. The image processoraccording to claim 9, wherein the detection section detects a triggerrelated to a distance between the user and the object.
 11. The imageprocessor according to claim 6, wherein the detection section detects atrigger related to a content type of the image.
 12. The image processoraccording to claim 1, further comprising a depth value acquisitionsection that acquires a depth value corresponding to the pixel viewed bythe user in the image on a basis of the visible pixel information andthe depth information.
 13. The image processor according to claim 1,wherein the pixel viewed by the user in the image is specified on abasis of information indicating a dominant eye of the user.
 14. Theimage processor according to claim 1, wherein the image is rendered froma viewpoint depending on a position or a posture of the user.
 15. Theimage processor according to claim 1, wherein, in a case where the userviews an image while moving, the predetermined period is decreased ascompared with a case where the user views an image while stopping. 16.An image processing method comprising: acquiring visible pixelinformation regarding a pixel viewed by a user in an image; and causinga processor to perform blur processing on the image on a basis of depthinformation indicating a depth value corresponding to each of pixels ofthe image, the blur processing being performed on the pixel viewed bythe user in the image on a basis of a delay depth value and the depthinformation during a predetermined period after the visible pixelinformation is acquired, the delay depth value being a delayed depthvalue corresponding to the pixel viewed by the user.
 17. A program thatcauses a computer to function as an image processor, the image processorincluding an acquisition section that acquires visible pixel informationregarding a pixel viewed by a user in an image, and an image processingsection that performs blur processing on the image on a basis of depthinformation indicating a depth value corresponding to each of pixels ofthe image, the image processing section performing the blur processingon a basis of a delay depth value and the depth information during apredetermined period after the visible pixel information is acquired,the delay depth value being a delayed depth value corresponding to thepixel viewed by the user.